Image processing apparatus and camera module

ABSTRACT

According to the Embodiments, an Image Processing apparatus includes a pixel interpolation processing unit. The pixel interpolation processing unit generates a sensitivity level value through addition of a first frequency range component of an image signal for a lacking color component and a second frequency range component of a frequency band lower than the first frequency range component. The pixel interpolation processing unit adjusts a ratio of the first frequency range component to be added to the second frequency range component.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2009-267352, filed on Nov. 25,2009; the entire contents of all of which are incorporated herein byreference.

FIELD

Embodiments described herein relate generally to an image processingapparatus and a camera module.

BACKGROUND

A so-called single-plate imaging apparatus, which is one of full-colorimaging apparatuses, is suitable for a case of requiring size reductionand cost reduction of a configuration such as a consumer digital stillcamera (DSC) and a camera-equipped cell-phone. In the single-plateimaging apparatus, any of color filters for red (R), green (G), and blue(B) is provided on a photoelectric element, and an image signal of aplurality of colors is obtained from one two-dimensional imaging elementby calculating sensitivity signals of lacking color components for eachpixel position. A sensitivity level value for the lacking colorcomponent is generated by interpolation processing that uses knownsensitivity level values at a target pixel and peripheral pixelstherearound (for example, see Japanese Patent Application Laid-open No.2001-197512).

Conventionally, in generating the sensitivity level value of the lackingcolor component, for example, a method is employed, in which alow-frequency component and a high-frequency component of image signalsare extracted and added. Degradation of resolution is suppressed by theaddition of the high-frequency component, so that it is possible toobtain a high-resolution image on which pixel interpolation processingis performed. However, in the case of requiring image processing thatfocuses on S/N (signal to noise ratio) more than the resolution, such asshooting under a low illumination environment, the high-frequencycomponent may become a factor of S/N degradation. Moreover, when animage signal is converted from an ROB format into a YUV format tooutput, a color noise may be degraded due to the high-frequencycomponent.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a cameramodule including an image processing apparatus according to a firstembodiment;

FIG. 2 is a diagram explaining an example of an embodiment ofinterpolation processing in a pixel interpolation processing unit;

FIG. 3 is a block diagram explaining an operation of the interpolationprocessing by the pixel interpolation processing unit;

FIG. 4 is a block diagram explaining an operation of the interpolationprocessing and a signal conversion in an image processing apparatusaccording to a second embodiment; and

FIG. 5 is a block diagram explaining an operation of the interpolationprocessing and the signal conversion in an image processing apparatusaccording to a third embodiment.

DETAILED DESCRIPTION

In general, according to one embodiment, an image processing apparatusincludes a pixel interpolation processing unit. The pixel interpolationprocessing unit generates a sensitivity level value of a lacking colorcomponent by interpolation processing of an image signal. The pixelinterpolation processing unit generates the sensitivity level valuethrough addition of a first frequency range component of the imagesignal for the lacking color component and a second frequency rangecomponent of a frequency band lower than the first frequency rangecomponent. The pixel interpolation processing unit adjusts a ratio ofthe first frequency range component to be added to the second frequencyrange component.

Exemplary embodiments of an image processing apparatus and a cameramodule will be explained below in detail with reference to theaccompanying drawings. The present invention is not limited to thefollowing embodiments.

FIG. 1 is a block diagram illustrating a configuration of a cameramodule including an image processing apparatus 1 according to a firstembodiment. The camera module includes the image processing apparatus 1,an imaging lens 2, and an image sensor 3. The imaging lens 2 captureslight from an object and forms an object image in a pixel unit 10 of theimage sensor 3. The image sensor 3 includes the pixel unit 10, ananalog-digital converter (ADC) 11, and a triangular-wave (VREF)generating circuit 12.

The pixel unit 10 images an object image by converting light from anobject into signal charges. The pixel unit 10 captures signal values ofR, G, and B in the order corresponding to a Bayer array and generatesanalog signals. The ADC 11 converts the analog signal from the pixelunit 10 into a digital signal (AD conversion). The ADC 11 can beintegrated with a column noise cancelling circuit (CDS). The VREFgenerating circuit 12 generates a VREF used for the AD conversion in theADC 11.

The image processing apparatus 1 performs various image processing to beexplained below on the digital image signal output from the ADC 11. Adefect correcting circuit 14 performs defect correcting processing forcorrecting a missing portion (defect) of the digital image signal due toa pixel that does not function normally in the pixel unit 10. A noisereducing circuit 15 performs noise reducing processing.

A pixel interpolation processing unit 17 performs pixel interpolationprocessing (demosaic processing) on the digital image signalstransmitted in the order of the Bayer array. A color-matrix processingunit 18 performs color-matrix operation processing (colorreproducibility processing) for obtaining color reproducibility. Acontour processing unit 19 performs contour enhancement processing byusing a correction coefficient calculated based on an imaging conditionby the image sensor 3 and a position of each pixel.

A gamma correction unit 20 performs gamma correction for correctingsaturation and brightness of an image. An edge extracting unit 21extracts an edge from the digital image signals transmitted in the orderof the Bayer array and outputs the extraction result to an RGB/YUVconverting unit 22. The RGB/YUV converting unit (signal converting unit)22 converts an image signal from an RGB format into a YUV format (forexample, YUV 422) by generating a luminance (Y) signal and a chrominance(UV) signal from sensitivity level signals of R, G, and B.

An AE/AWB operation circuit 23 calculates each coefficient for AE (autoexposure) and AWB (auto white balance) from the result of the gammacorrection by the gamma correction unit 20 and output them. A digitalAMP coefficient circuit 24 calculates a digital AMP coefficient based onthe output of the AE/AWB operation circuit 23 and a shading correctioncoefficient. An analog gain (AG) setting unit 25 sets AG based on theoutput of the AE/AWB operation circuit 23. The AG set in the AG settingunit 25 is input to the VREF generating circuit 12 and the pixelinterpolation processing unit 17. The VREF generating circuit 12generates the VREF in accordance with the AG from the AG setting unit25.

Line memories 13 and 16 temporarily store therein data on the digitalimage signals transmitted in the order corresponding to the Bayer array.The defect correcting circuit 14 and the noise reducing circuit 15 sharethe line memory 13. The image processing apparatus 1 performs digitalgain AMP processing for the AE, the AWB, and a lens shading correctionon the image signals from the defect correcting circuit 14 and the noisereducing circuit 15 by the digital AMP coefficient from the digital AMPcoefficient circuit 24. The image signal on which the digital gain AMPprocessing is performed is stored in the line memory 16.

The image processing apparatus 1 sequentially performs respectiveprocessing from the pixel interpolation processing unit 17 to theRGB/YUV converting unit 22 on the image signal stored in the line memory16 and outputs the image signal converted into the YUV format in theRGB/YUV converting unit 22. The configuration of the image processingapparatus 1 explained in the present embodiment is only an example andcan be appropriately modified. For example, in the configurationexplained in present embodiment, change, such as addition of an elementfor other processing and omission of an optional element, can be made.

FIG. 2 is a diagram explaining an example of an embodiment of theinterpolation processing in the pixel interpolation processing unit 17.An R pixel is a pixel that is provided with a color filter thatselectively transmits R light, in which R is a known color component andG and B are lacking color components. A G pixel is a pixel that isprovided with a color filter that selectively transmits G light, inwhich G is the known color component and R and B are the lacking colorcomponents. A B pixel is a pixel that is provided with a color filterthat selectively transmits B light, in which B is the known colorcomponent and R and G are the lacking color components. In the pixelunit 10, the R pixels, the G pixels, and the B pixels are arranged inthe Bayer array.

As shown in FIG. 2, 25 pixels D1 to D25 are arranged in a matrix mannerof 5×5. When the center pixel D13 of the 25 pixels D1 to D25 is the Rpixel, the pixel interpolation processing unit 17 calculates eachsensitivity level value of R, G, and B for the pixel D13, for example,by the following equation. The sensitivity level values of G and B asthe lacking color components of the pixel D13 are generated by theinterpolation processing using the sensitivity level values of the knowncolor components. In the following equation, D1, D2, . . . , and D25represent the sensitivity level values of the known color componentsdetected in the pixel D1, the pixel D2, . . . , and the pixel D25,respectively.R=D13G=(D8+D12+D14+D18)/4+{D13−(D3+D11+D15+D23)/4}/4B=(D7+D9+D17+D19)/4+{D13−(D3+D11+D15+D23)/4}/2

In the interpolation equation used for G, the former term(D8+D12+D14+D18)/4 corresponds to the low-frequency component extractedfrom the image signals and {D13−(D3+D11+D15+D23)/4} in the latter termcorresponds to the high-frequency component extracted from the imagesignals. In the interpolation equation used for B, the former term(D7+D9+D17+D19)/4 corresponds to the low-frequency component extractedfrom the image signals and {D13−(D1+D11+D15+D23)/4} in the latter termcorresponds to the high-frequency component extracted from the imagesignals. The pixel interpolation processing unit 17 generates thesensitivity level value through addition of the high-frequency componentthat is a first frequency range component of the image signals for thelacking color component and the low-frequency component that is a secondfrequency range component of a frequency band lower than the firstfrequency range component.

FIG. 3 is a block diagram explaining an operation of the interpolationprocessing by the pixel interpolation processing unit 17. The linememory 16 holds the digital image signals for four lines (4H). In thepixel interpolation processing unit 17, data for totally five lines,i.e., the four lines held in the line memory 16 and one line immediatelybefore being input to the line memory 16, is input.

The pixel interpolation processing unit 17 samples the signal values ofthe data input on the five lines in accordance with a VH count, andgenerates each sensitivity level value of the R component, the Gcomponent, and the B component. The pixel interpolation processing unit17 applies the interpolation equation similar to the above to each colorcomponent to extract the low-frequency component and the high-frequencycomponent for each of the R component, the G component, and the Bcomponent.

The pixel interpolation processing unit 17 multiplies the high-frequencycomponent of each color component by the AG coefficient before addingthe low-frequency component and the high-frequency component for each ofthe R component, the G component, and the B component. The AGcoefficient is a coefficient that is correlated with an analog gain ofthe image signal set in the AG setting unit 25. The AG coefficient isset to have a correlation so that the AG coefficient is linearly loweredas the AG becomes high. In the image processing apparatus 1, thecorrelation between the AG and the AG coefficient is stored in advance.The pixel interpolation processing unit 17 multiplies the high-frequencycomponent by the AG coefficient obtained by referring to the correlationwith respect to the AG input from the AG setting unit 25. The pixelinterpolation processing unit 17 adjusts the ratio of the high-frequencycomponent to be added to the low-frequency component by themultiplication of the AG coefficient. The pixel interpolation processingunit 17 outputs the sensitivity level value of each color componentgenerated through addition of the low-frequency component and thehigh-frequency component that is multiplied by the AG coefficient.

The high-frequency component is multiplied by the AG coefficient havinga correlation so that the AG coefficient is lowered as the AG becomeshigh, so that the pixel interpolation processing unit 17 adjusts toreduce the ratio of the high-frequency component in the condition inwhich the AG is made high such as shooting under a low illuminationenvironment. Therefore, in the case of requiring the image processingthat focuses on S/N more than resolution, it becomes possible to reducethe ratio of the high-frequency component to be a factor of degradationof the S/N. The sensitivity level value is generated through addition ofthe high-frequency component of which ratio is adjusted in accordancewith the AG, so that it also becomes possible to obtain the effect ofsuppressing degradation of the resolution. Thus, the image processingapparatus 1 can suppress degradation of the resolution in the case ofgenerating the sensitivity level value of the lacking color component bythe interpolation processing of the image signal and suppressdegradation of the S/N.

The interpolation processing applied in the present embodiment can useany method in which extraction of the high-frequency component and thelow-frequency component of the image signals is performed, and theinterpolation equation can be appropriately modified. The number oflines of data input for the interpolation processing is not limited tothe case of five lines explained in the present embodiment and can beany number.

The AG coefficient is not limited to the case of having the correlationto linearly increase or decrease with respect to the AG, and it issufficient that the AG coefficient is set to become a low value withrespect to the high AG requiring improvement of the S/N. For example, aninverse number of the AG can be employed as the AG coefficient.Moreover, it is applicable that the pixel interpolation processing unit17 adjusts the ratio of the high-frequency component with respect to theAG only in a predetermined range in which the image processing focusingon the S/N more than the resolution is required.

FIG. 4 is a block diagram explaining an operation of the interpolationprocessing and a signal conversion in an image processing apparatusaccording to a second embodiment. The present embodiment ischaracterized in that the chrominance (UV) signal is generated from thelow-frequency components of the image signals in the signal conversionin the RGB/YUV converting unit 22. Components that are the same as thosein the first embodiment are given the same reference numerals andoverlapping explanation is appropriately omitted.

The pixel interpolation processing unit 17 extracts the low-frequencycomponent and the high-frequency component for each of the R component,the G component, and the B component and generates the sensitivity levelvalue of the lacking color component by addition of the low-frequencycomponent and the high-frequency component. The RGB/YUV converting unit22 generates a Y signal from the low-frequency components and thehigh-frequency components that are extracted and added in the pixelinterpolation processing unit 17 and outputs the sensitivity level valueof the Y component. Moreover, the RGB/YUV converting unit 22 generatesthe UV signal only from the low-frequency components from among thelow-frequency components and the high-frequency components that areextracted in the pixel interpolation processing unit 17 and outputs thesensitivity level values of the U component and the V component.

In this manner, the UV signal is generated from the ROB signal fromwhich the high-frequency component is removed, so that degradation ofthe S/N ratio related to chromaticity can be suppressed. In view of thefact that people are more sensitive to change in the luminance thanchange in the chromaticity, degradation of the resolution can besuppressed by generating the Y signal from the ROB signal in which thelow-frequency component and the high-frequency component are added.Therefore, in the present embodiment again, it becomes possible tosuppress degradation of the resolution in the case of generating thesensitivity level value of the lacking color component by theinterpolation processing of the image signal and suppress degradation ofthe S/N.

FIG. 5 is a block diagram explaining an operation of the interpolationprocessing and the signal conversion in an image processing apparatusaccording to a third embodiment. The present embodiment is characterizedin that characteristics of the first embodiment and the secondembodiment are combined. Components that are the same as those in thefirst embodiment and the second embodiment are given the same referencenumerals and overlapping explanation is appropriately omitted.

The pixel interpolation processing unit 17 multiplies the high-frequencycomponent of each color component by the AG coefficient before addingthe low-frequency component and the high-frequency component for each ofthe R component, the G component, and the B component. The AGcoefficient is set to have a correlation so that the AG coefficient islinearly lowered as the AG becomes high. The pixel interpolationprocessing unit 17 outputs the sensitivity level value of each colorcomponent generated through addition of the low-frequency component andthe high-frequency component that is multiplied by the AG coefficient.

The RGB/YUV converting unit 22 generates the Y signal from thelow-frequency components and the high-frequency components that areextracted and added in the pixel interpolation processing unit 17 andoutputs the sensitivity level value of the Y component. Moreover, theRGB/YUV converting unit 22 generates the UV signal only from thelow-frequency components from among the low-frequency components and thehigh-frequency components that are extracted in the pixel interpolationprocessing unit 17 and outputs the sensitivity level values of the Ucomponent and the V component.

In the present embodiment again, it becomes possible to suppressdegradation of the resolution in the case of generating the sensitivitylevel value of the lacking color component by the interpolationprocessing of the image signal and suppress degradation of the S/N.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

1. An image processing apparatus comprising: a pixel interpolationprocessing unit that generates a sensitivity level value of a lackingcolor component by interpolation processing of an image signal, whereinthe pixel interpolation processing unit generates the sensitivity levelvalue through addition of a first frequency range component of the imagesignal for the lacking color component and a second frequency rangecomponent of a frequency band lower than the first frequency rangecomponent, and adjusts a ratio of the first frequency range component tobe added to the second frequency range component, and the pixelinterpolation processing unit adjusts the ratio of the first frequencyrange component by multiplication of an analog gain coefficient that iscorrelated with an analog gain of the image signal.
 2. The imageprocessing apparatus according to claim 1, wherein the analog gaincoefficient is set to have a correlation so that the analog gaincoefficient is lowered as the analog gain becomes high.
 3. The imageprocessing apparatus according to claim 1, further comprising a signalconverting unit that converts a sensitivity level signal for each colorcomponent into a chrominance signal and a luminance signal, wherein thesignal converting unit generates the luminance signal from the firstfrequency range component of which ratio is adjusted in the pixelinterpolation processing unit and the second frequency range component.4. The image processing apparatus according to claim 3, wherein thesignal converting unit generates the chrominance signal from the secondfrequency range component.
 5. An image processing apparatus comprising:a pixel interpolation processing unit that generates a sensitivity levelvalue of a lacking color component by interpolation processing of animage signal; and a signal converting unit that converts a sensitivitylevel signal for each color component into a chrominance signal and aluminance signal, wherein the pixel interpolation processing unitgenerates the sensitivity level value through addition of a firstfrequency range component of the image signal for the lacking colorcomponent and a second frequency range component of a frequency bandlower than the first frequency range component, the pixel interpolationprocessing unit adjusts a ratio of the first frequency range componentto the second frequency range component used for generation of theluminance signal, the pixel interpolation processing unit adjusts theratio of the first frequency range component by multiplication of ananalog gain coefficient that is correlated with an analog gain of theimage signal, and the signal converting unit generates the chrominancesignal from the second frequency range component of the image signal. 6.The image processing apparatus according to claim 5, wherein the signalconverting unit generates the luminance signal from the sensitivitylevel signal in which the first frequency range component and the secondfrequency range component are added.
 7. The image processing apparatusaccording to claim 5, wherein the analog gain coefficient is set to havea correlation so that the analog gain coefficient is lowered as theanalog gain becomes high.
 8. A camera module comprising: a pixelinterpolation processing unit that generates a sensitivity level valueof a lacking color component by interpolation processing of an imagesignal, wherein the pixel interpolation processing unit generates thesensitivity level value through addition of a first frequency rangecomponent of the image signal for the lacking color component and asecond frequency range component of a frequency band lower than thefirst frequency range component, and adjusts a ratio of the firstfrequency range component to be added to the second frequency rangecomponent, and the pixel interpolation processing unit adjusts the ratioof the first frequency range component by multiplication of an analoggain coefficient that is correlated with an analog gain of the imagesignal.
 9. The camera module according to claim 8, wherein the analoggain coefficient is set to have a correlation so that the analog gaincoefficient is lowered as the analog gain becomes high.
 10. The cameramodule according to claim 8, further comprising a signal converting unitthat converts a sensitivity level signal for each color component into achrominance signal and a luminance signal, wherein the signal convertingunit generates the luminance signal from the first frequency rangecomponent of which ratio is adjusted in the pixel interpolationprocessing unit and the second frequency range component.
 11. The cameramodule according to claim 10, wherein the signal converting unitgenerates the chrominance signal from the second frequency rangecomponent.
 12. The camera module according to claim 8, furthercomprising: a pixel unit that images an object image; an analog-digitalconverter that converts an analog signal from the pixel unit into adigital signal; and a triangular-wave generating circuit that generatesa triangular wave used in conversion from the analog signal into thedigital signal in the analog-digital converter, wherein thetriangular-wave generating circuit generates the triangular wave inaccordance with an analog gain of the image signal.